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NASA Langley Research Center conducted flight tests at the Eagle County, Colorado airport to evaluate 
synthetic vision concepts. Three display concepts (size “A” head-down, size “X” head-down, and head- 
up displays) and two texture concepts (photo, generic) were assessed for situation awareness and flight 
technical error / performance while making approaches to Runway 25 and Runway 07 and simulated 
engine-out Cottonwood 2 and KREMM departures. The results of the study confirm the retrofit 
capability of the HUD and Size “A” SVS concepts to significantly improve situation awareness and 
performance over current EFIS glass and non-glass instruments for difficult approaches in terrain- 
challenged environments. 


INTRODUCTION 

The Synthetic Vision Systems (SVS) element of the National 
Aeronautics and Space Administration’s (NASA) Aviation 
Safety Program (AvSP) is striving to eliminate poor visibility 
as a causal factor in aircraft accidents as well as enhance 
operational capabilities of all aircraft. To accomplish these 
safety and situation awareness improvements, the SVS concept 
is designed to provide a clear view of the world ahead through 
the display of computer generated imagery derived from an 
onboard database of terrain, obstacle, and airport information. 

The ability of a pilot to ascertain critical information through 
visual perception of the outside environment can be limited by 
various weather phenomena, such as rain, fog, and snow. 
Since the beginning of flight, the aviation industry has 
continuously developed various devices to overcome low- 
visibility issues, such as attitude indicators, radio navigation, 
and instrument landing systems. Recent advances include 
moving map displays, incorporating advances in navigational 
accuracies from the Global Positioning System, and enhanced 
ground proximity warning systems. However, all of the 
aircraft information display concepts developed to date require 
the pilot to perform various additional levels of mental model 
development and maintenance and information decoding in a 
real-time environment when outside visibility is restricted. 

Better pilot situation awareness during low visibility 
conditions can be provided by SVS displays. New 
technological developments in navigation performance, low- 
cost attitude and heading reference systems, computational 
capabilities, and graphical displays allow for the prospect of 
SVS displays for virtually all aircraft classes. SVS display 
concepts employ computer-generated terrain imagery, on- 
board databases, and precise position and navigational 
accuracy to create a three dimensional perspective presentation 
of the outside world, with necessary and sufficient information 
and realism, to enable operations equivalent to those of a 
bright, clear, suimy day regardless of the outside weather 
condition. The safety outcome of SVS is a display that should 
help reduce, or even prevent, controlled-flight-into-terrain 
(CFIT), which is the single greatest contributing factor to fatal 
worldwide airline and general aviation accidents. Other safety 


benefits include reduced runway incursions and loss-of-control 
accidents (Prinzel et al, 2000; Williams et al, 2001) in 
addition to significant economic benefits (Hemm, 2000). 

SVS Research Issues 

To provide a better definition of the concept of operations 
(CONOPS) of synthetic vision technology for commercial and 
business aircraft, a workshop resulting in a CONOPS 
document was held in early 2000 (Williams, et. al, 2001). The 
focus of this event was to obtain wide ranging input on the 
benefits and features which synthetic vision might incorporate. 
This meeting included representatives from NASA, DoD, 
FAA, industry professional organizations, pilots, airlines, 
aircraft and avionics manufacturers, airports, and academic 
institutions. The result of the workshop was a “shopping list” 
of research issues that need to be explored in developing SVS 
display concepts. 

NASA Langley Research Center Research (LaRC) in 
Hampton, VA has conducted past research examining many of 
the issues that were identified in the CONOPS SVS workshop. 
Comstock, Glabb, Prinzel, & Elliot (2001) and Stark, 
Comstock, Prinzel, Burdette, and Scerbo (2001) focused on the 
issues of retro- and forward-fit issues of display size and field- 
of-view (FOV) for current and future transport aircraft. The 
objective of those studies was to examine whether a SVS 
display could be retrofitted into Electronic Flight Instruments 
(EFIS) Size “A” (e.g., B-757-200) Electronic Attitude 
Direction Indicator (EADI) and Size “D” (e.g., B-777) Primary 
Flight Display (PFD). A size “X” head-down display (SVS- 
HDD) was also tested that would represent the display real 
estate that may be found on future aircraft. The results of those 
studies confirm that a SVS display could be incorporated as 
part of an EFIS suite and replace an EADI or PFD in providing 
a perspective SVS display. Furthermore, although all pilots 
preferred the larger display, the FOV on a smaller SVS display 
could be adjusted to minimize situation awareness or pilot 
performance penalty costs. Regardless of FOV or display size, 
pilots reported greater situation awareness (i.e., self-report, 
SA-SWORD) and had lower flight technical error (FTE) while 
making approaches to a non-precision runway (AVL) with the 
SVS display compared to the traditional EADI. 



The Current Study 

Previous research was conducted in a fixed-based simulator at 
NASA LaRC. Although the research data indicates that an 
SVS display could significantly enhance situation awareness in 
terrain-challenged environments, these results would need to 
be confirmed in real-world flight tests. Data was collected 
during a 2000 NASA flight test on a NASA B-757-200 at the 
Dallas / Fort-Worth (DFW) airport. However, the objective of 
that flight test was to examine the efficacy of SVS for making 
approaches to an operationally challenged environment and not 
to determine the safety benefits of SVS for terrain awareness. 
Therefore, a NASA B-757 flight test was conducted in the 
Eagle County, CO Regional Airport local area (FAA airport 
identifier: EGE) to evaluate tactical SVS display concepts in a 
terrain-challenged operational environment. 

EGE represents an ideal location to test the effectiveness of 
SVS technologies for terrain awareness and separation for 
approaches that puts the aircraft close to mountainous terrain. 
There are 2 approaches that are available for use by the major 
airlines flying 757s into the EGE airport: FMS 25 and FMS 07 
approaches. Although the runway 25 approach is the most 
common approach, tailwind restrictions can require the need to 
make the circle-to-land 07 approach. However, as a pilot from 
American Airlines noted, the approach has rarely been made 
because of the close proximity it places the aircraft to the 
surrounding terrain. Furthermore, traffic into EGE is heaviest 
during the skiing season in which Instrument Meteorological 
Conditions are prevalent. The EGE airport has a special 
airport status and special procedures and training have been 
approved by the FAA for the 07 approach. The approach 
requires Visual Meteorological Conditions (VMC) that 
significantly limits the use of this end of the runway. SVS 
could potentially enhance the operational capability of EGE, 
and similar airports, in which terrain and obstacles limits IMC 
approaches. Therefore, the EGE area was selected to 
demonstrate the efficacy of SVS to maintain required 
navigation performance (RNP) and terrain separation during 
the circle-to-land 07-runway approach during Instrument 
Flight Rules (IFR) conditions, and confirm past simulator 
research. 

Objectives 

• Confirm the capability of SVS head-down display as 
retrofit concepts in glass cockpits across various SVS 
display sizes (Size A, Size X) 

• Examine the usability / acceptability and situation 
awareness benefits of different methods of photo-textured 
and generic -textured terrain database concepts 

• Determine FOV requirements for different size SVS 
display concept during different segments of approach to 
EGE 

• Assess the safety benefits and actual navigation 
performance of the SVS concept during simulated 
Instrument Meteorological Conditions (IMC) 

• Evaluate the use of Head-Up Displays (HUD) as a retrofit 
concept for non-glass cockpits 


• Assess flight technical error (FTE) during manually flown 
landing approaches and go-around maneuvers with the 
SVS technologies for both runway 25 and the more 
operationally challenging runway 07 and verify RNP 
capabilities 

METHOD 

Experimental Design 

A 3 display condition (A, X, HUD) X 2 texture (generic, 
photo) X 2 procedure (runways 25 & 07) mixed- factorial 
design was used. The flight test was designed to make 
comparisons across these display concepts as well as to a 
baseline EADI with a simulated Terrain Awareness and 
Warning System (TAWS). Dependent variables included 
flight technical error (FTE), run questionnaires, semi- 
stmctured interviews, verbal protocols, and SA-SWORD. 

Evaluation Pilots 

Six evaluation pilots, representing 3 airlines, FAA, and 
Boeing, flew 12 research flights totaling 51.6 flight hours. 
Eighty-Seven flight test runs were conducted to evaluate the 
NASA display concepts with 52 being flown to Runway 07 
and 35 flown to Runway 25. All participants were ATP-rated 
B-757 captains with operating experience at the EGE airport. 
Prior to deployment, all evaluation pilots received a one-day 
training course at LaRC to familiarize them with the SVS 
display concepts. 

Elight Test Equipment 

The flight test was conducted using the NASA LaRC B-757- 
200 aircraft. The evaluation pilot occupied the left seat and the 
safety pilot occupied the right seat. Installed over the 
traditional EFIS instruments, a SVS research display (SV-RD) 
and an overhead Dassault projection HUD, driven by a Flight 
Dynamics Head-Up Guidance System (HGS) 4000 HUD 
computer, was installed on the Captain’s side. The SVS-RD is 
a commercially available 18.1” LCD monitor and had 1280 X 
1024 horizontal pixel resolution with 900 nits of brightness. 
The HGS-4000 computer provides raster capability for terrain 
texturing and tunnel symbology while retaining high-quality 
stroke symbology for primary flight information (e.g., 
airspeed, altitude). The “background raster” was the SVS 
terrain; the “foreground raster” consisted of the flight 
instruments and tunnel (“highway-in-the-sky”) symbology. 
The FOV of the HUD was 22° vertical and 28° horizontal. 
Figure 1 & 2 shows the SVS-HDD and SVS-HUD concepts, 
respectively. A vision restriction device covered the left-seat 
forward windscreen to simulate Instrument Meteorological 
Conditions (IMC). Also, brightness controls and a de-clutter 
HUD switch was available to the evaluation pilots. The pilot, 
through an interface on the center pedestal, could change FOV 
settings: Unity, 30 deg, 60 deg, and 90 deg. The SVS concepts 
were generated using an Integraph ZxlO computer that 
provided video updates of > 30 Hz and symbology was 
generated using OpenGL graphics software programming 
language. 




Figure 1. The SVS-Head Down Display Concept 


Terrain Database 

A digital elevation model (DEM) of 100 nm by 100 nm was 
used through the use of multi-resolution post-spacing varying 
between 1 and 3 arc-seconds and retaining a vertical accuracy 
of between 15 and 30 meters. The SVS photo-texture concept 
used multi-resolution imagery of between 1 and 32 
meters/pixel to complete the DEM overlay and color balancing 
was used to create the look of photo-realism. The generic- 
texture concept used color mapping technique, known as 
“elevation shading”, that used Aeronautical Chart legends to 
create the color elevation range through the use of 12 color 
bands segmented into 250 meter ranges. The SVS-HUD 
concept for generic only used the green channel and varied the 
color intensities to map changes in elevation. The photo- 
texture HUD concept used the RGB file format from the head- 
down photo-texture concept, masked out the red and blue 
channels, and converted the image back to “ECW” format. 

Evaluation Tasks 

The EGE airport has a “Special Airport” designation from the 
FAA and precision approach landing aids are not available 
because of terrain. Only an offset localizer with a DME is 
available to provide support for Localizer-DME approaches 
(EDA). Special Flight Management System (FMS)-based 
approach and landing procedures and training have been 
developed and certified for EGE. The evaluation tasks were 
developed based on these existing FMS approach procedures 
and comprised the FMS Runway 25 approach and 
Cottonwood-2 departure and the FMS Runway 25 approach 
with Visual Arrival to Runway 07 and KREMM Departure 
(Figure 4). The flight test simulated engine-out procedures 
during the departures by reducing 
thrust on both engines equally to 
approximate single-engine climb 
performance. A “follow-me” 
aircraft (FMA; figure 3) was 
resident on the SVS display that 
pilots used to train the velocity 
vector on the path guidance 
symbology to maintain vertical 
and lateral path guidance. 



Figure 2. The SVS Head-Up Display Concept 


RESULTS 

A significant amount of research data was collected and 
analyzed and space limitations do not permit full reporting 
here. For further detail, please consult the NASA Technical 
Paper (TP) that discusses all the experimental results. 
Highlights of the data are described below. 




Figure 4. Runway 07 & 25 Approaches & Departures 


Post-Run Questionnaires 

Post-run questions were asked of each evaluation pilot to 
assess terrain awareness, while flying the approaches with the 
display concepts. The most significant result was that 
participants evaluated the Size A Photo and Size X Generic 
and Photo concepts to be significantly better in terms of terrain 
awareness over the traditional EADI, F(6, 73) = 2.69, p < .01. 

SA-SWORD 

Pilots were asked to complete a paired-comparison SA- 
SWORD (Vidulich & Hughes, 1991). The responses were 
averaged and the overall rankings were: Photo Size X, Generic 











Size X, Photo HUD, Generic HUD, Photo Size A, Generic 
Size A, and Baseline EADI. The SA-SWORD data was 
analyzed, F(6, 18) = 6.98, g < .01, and revealed significant 
pairwise comparisons for Size X Photo and Size X Generic. 
Size X Photo had significantly higher SA-SWORD ratings 
than all other SVS display concepts except for Size X Generic. 
The Size X Generic was significantly higher than all SVS 
display concepts with the exception of the HUD Photo. 

Semi-Structured Interview 

At the completion of each flight test day, all evaluation pilots 
participated in a semi-structured interview and completed a 
post-experimental questioimaire packet. Because of the 
number of questions asked, the complete data set cannot be 
addressed here. An example of a question asked: “Did the use 
of the NASA HUD improve your situation awareness beyond 
the use of the EADI w/ flight director for approach to rwy. 07? 
If so, please rate how much more your level of situation 
awareness was enhanced.” The average rating (from 0 to 9), 
reflecting SA enhancement, was M = 7.86, p < .001 . Similar 
responses were found for Size A (M = 7.14) and Size X (M = 
8.0) compared to EADI w/ flight director baseline condition. 
Therefore, the SVS concepts were reported to provide a 
significant enhancement of situation awareness than that 
provided by the traditional EADI with flight director and 
TAWS. 

Over 80 semi-structured interview questions were asked of 
pilot subjects covering a number of operational and 
implementation issues. None of the seven pilots reported any 
negative responses directed toward the general SVS concept. 
One interesting finding concerned the use of the “follow me 
aircraft” guidance and velocity vector. Participants were asked 
to rate the ease of predicting flight path using these guidance 
cues for each of the SVS concepts; the average rating was 7.5 / 
9.0, p < .001. Follow-on comments revealed that the coupled 
symbology allowed the pilots to remain “ahead of the aircraft” 
and better scan the SVS and other instruments to maintain their 
situational model. Moreover, pilots noted that maintaining 
flight path was easier with these two sets of guidance than 
having only a flight path tunnel. However, regardless of 
whether the guidance was used with or without the tunnel, 
pilots reported a significant enhancement of navigation 
awareness than that provided for by just the traditional EFIS 
instruments. 

Pilot Comments 

Video and audio recordings of each pilot were taken while they 
flew the two approaches and departures. These were intended 
to capture the verbal protocols and to verify data collection of 
post-run questions. Overall, pilot comments were very 
positive of the SVS concept. An example comment for Size A 
Photo Realistic was, 

“I often commented to people over the years that I 
never ever flew a circling approach that I was ever 
comfortable with particularly at night. This was the first 
time I ever had an occasion of circling an approach with 
the kind of information I would love to have in a 
circling approach. Keeping me safe, I could see the 


terrain, taking me where I want to go, getting me all 
t5fpes of information in terms of where I am relative to 
the end of the runway. I mean it’s the best of all 
possible worlds in terms of safety.“ 

The comment reflects the majority of the comments, from 
highly experienced 757 line pilots that have flown the EGE 
approaches, that the SVS display concepts significantly 
enhanced situation awareness, especially for the workload 
intensive 07 approach. The most significant negative 
comments concerned the opaque nature of the SVS HUD 
concept and that pilots should have the ability to de-clutter 
(which was possible in this experiment but only for the HUD) 
the symbology and the terrain depiction for all SVS display 
concepts. Another significant issue for pilots was the 

presentation of obstacle data and the updating of the navigation 
database to ensure data integrity (NASA’s goal is integrity 
level of 10'®). 

Evaluation Pilot Performance 

Constraints beyond the control of the authors have limited 
access to performance data available at time of publication 
(June 2002). Therefore, only the lateral path error component 
of flight technical error of the approach is reported here. 
However, the complete “path steering error” of actual 
navigation performance (Figure 5) and missed approached data 
are to be discussed at the HFES conference and results will be 
documented in the NASA TP final report. 

Approach runs were divided into segments, graphically 
presented in Figure 6, and segments 1 , 2, 7, and 8 comprise the 
07 approach and segments 1 - 5 represent the 25 approach. 
Segment 6 was not analyzed because the safety pilot had 
control of the aircraft. An ANOVA found no significant 
differences for texture (photo, generic) or display type (SVS 
only) for the 25 approach, p > .05. Furthermore, no effect was 
found for texture concept for 07 approach, p > .05. 

Significant results, however, were found for lateral RMS for 
the 07 circle-to-land approach scenario for segments 2, 7, and 
8. No significant result was found for segment 1 between the 
EADI and the SVS concepts, F (2, 10) = 9.747, p >.05. 
Therefore, pilots were stabilized on approach before Talia 
regardless of display condition flown. However, after reaching 
Talia and lining up for approach (segment 2), significant 
differences in performance was found, F (3, 12) = 16.810, p < 
.05. Such a result was evident throughout the 07 approach for 
segments 7, F (3, 12) = 17.80, p_ < .05, and 8, F (3, 12) = 
110.05, p_ < .05. Overall, the EADI delta, from SVS lateral 
RMSE, averaged 853 feet. Table I shows the mean values for 
the display concepts across segments. 
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Figure 5. Required Navigation Performance Components 


Table 1. Mean RMS Across 07 Approach Segments 


Segment 

EADI 

Size A 

SizeX 

HUD 

1 

290ft 

105 ft 

97ft 

212ft 

2 

1106ft 

49ft 

31ft 

49ft 

7 

658ft 

36ft 

36ft 

49ft 

8 

960ft 

87ft 

97ft 

58ft 



1-2: Both 
3-5: 25 Only 
6-8: 07 Only 


Coaonwood-2 


Figure 6. Approach Segments for FMS 25 & 07 Analyses 


The results confirm the hypotheses that Synthetic Vision 
provides improved path control and situation awareness in 
terrain-challenged airport operating environments, and that 
retrofit into glass and non-glass cockpits is viable and will 
provide safety and performance benefits. No significant 
differences in performance were found between the texture 
concepts although pilots reported subjective preference for 
photo realistic for improved SA. Pilots also reported 
preferences for larger SVS display sizes, but again no 
performance differences were found. The finding may reflect 
the highly skilled abilities of commercial pilots to fly the 
approach with precision regardless of the SVS concept. 
However, unlike the 25 approach, the 07 EGE approach is a 
high workload circling maneuver and RMSE for the SVS 
concepts was significantly better than the EADI w/ flight 
director. Moreover, 100% of the pilots expressed an 
overwhelming preference and SA improvement for the SVS 
concepts over the EADI. 

A number of issues, however, were uncovered and these 
included those regarding the use of FOV settings; minification; 
opaqueness of the HUD; system accuracy and integrity; 
human-system interaction concerns, such as trust, 
complacency, tunneling, and compellingness. These issues are 
part of the human factors list of concerns being addressed by 
NASA and industry partners and will be researched in an 
upcoming 757 flight test and ongoing laboratory research at 
NASA LaRC. Despite these reservations, the results of the 
EGE flight test confirm the findings of simulator experiments 
demonstrating the potential of SVS to significantly mitigate, or 
even eliminate, CFIT accidents. The SVS concepts described 
herein should be considered as part of a suite of technologies 
being developed (e.g., runway incursion monitors, enhanced 
sensors) that, together coupled with the SVS database and 
displays, represents the SVS concept and could considerably 
help meet the national aviation safety goal of reducing the 
accident rate by lOx within 25 years (NASA, 1999). 
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